<html>
<head><meta charset="utf-8"><title>Runners for custom test annotations · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html">Runners for custom test annotations</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="186330785"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186330785" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186330785">(Jan 22 2020 at 21:05)</a>:</h4>
<p>There exists a number of custom test annotations:  <code>#[async_std::test]</code>, <code>#[actix_rt::test]</code>, <code>#[tokio::test]</code>.</p>
<p>Currently RA does not provide any runnables for those, but at least for those 3 it's the same Cargo command used to run them: you can check it by removing the <code>tokio::</code> prefix first to force the runneer to appear, run it, return the prefix and rerun the task: the test will run.</p>
<p>Would it make sense to detect those and create runners for them? <br>
We can look for attributes ending with <code>::test</code> and create a runner for every one found.</p>



<a name="186330990"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186330990" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186330990">(Jan 22 2020 at 21:07)</a>:</h4>
<p>There are, of course, attributes that require a different command to run the tests, for example, the <code>#[wasm_bindgen_test]</code> one, so we might eventually produce some false-positives, but that seems better than no way to run the tests at all.</p>
<p>Thinking of alternative ways to achieve this, we can add some setting with a list of attribute values to look for to create the runners for them.</p>



<a name="186334156"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186334156" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> std::Veetaha <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186334156">(Jan 22 2020 at 21:44)</a>:</h4>
<p>To clarify, are you talking about this button over there?<br>
<a href="/user_uploads/4715/_J0wg30Y_3dd847AhqzAarOl/pasted_image.png" target="_blank" title="pasted_image.png">pasted image</a></p>
<div class="message_inline_image"><a href="/user_uploads/4715/_J0wg30Y_3dd847AhqzAarOl/pasted_image.png" target="_blank" title="pasted image"><img src="/user_uploads/4715/_J0wg30Y_3dd847AhqzAarOl/pasted_image.png"></a></div>



<a name="186344746"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186344746" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186344746">(Jan 22 2020 at 23:59)</a>:</h4>
<p>I think just checking "if it ends with test" should be good enough!</p>



<a name="186371398"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186371398" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186371398">(Jan 23 2020 at 09:30)</a>:</h4>
<blockquote>
<p>To clarify, are you talking about this button over there?</p>
</blockquote>
<p>Yes, this is what I want to have for various <code>#[whatever::test]</code> annotated methods.</p>
<blockquote>
<p>I think just checking "if it ends with test" should be good enough!</p>
</blockquote>
<p>Great, my thoughts also. <br>
I'm not entirely sure how to extract the info I need from the <code>ast::Attr</code>, but I'll work on that and find a way hopefully.</p>



<a name="186372869"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186372869" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Wojciech Polak <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186372869">(Jan 23 2020 at 09:54)</a>:</h4>
<p>Hey, I'm maintainer of <code>test-case</code> (<a href="https://github.com/frondeus/test-case" target="_blank" title="https://github.com/frondeus/test-case">https://github.com/frondeus/test-case</a>)</p>
<p>My macros are <code>#[test_case(...)]</code> which are expanded into one or more functions with <code>#[test]</code> attribute.<br>
It'd be great if RA could support not only <code>.*test</code> but also <code>test.*</code>. Maybe <code>.*test.*</code>?</p>



<a name="186374079"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186374079" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186374079">(Jan 23 2020 at 10:09)</a>:</h4>
<p>Sounds reasonable, I'll try to come up with a better heuristics.</p>



<a name="186374712"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186374712" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> std::Veetaha <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186374712">(Jan 23 2020 at 10:18)</a>:</h4>
<p>I bet we can benefit from the experience of other tools. For example, I know that Java extension provides an ability to run and debug a single JUnit test from VSCode. I don't believe that extension tries to heuristically determine tests, but rather I suppose they just integrate with major test frameworks.</p>



<a name="186375266"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186375266" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Florian Diebold <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186375266">(Jan 23 2020 at 10:25)</a>:</h4>
<p>the long-term approach would be to expand the attribute macros and detect test cases in the expanded code, but we're not there yet</p>



<a name="186375792"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186375792" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186375792">(Jan 23 2020 at 10:32)</a>:</h4>
<p>I think waiting for the macros expansion would be the best way, since it's more generic.</p>
<p>I've had a quick glance and the way the JUnit tests are supported in the RedHat plugin and it looks like the VSCode extension needs another one to run the tests: <a href="https://github.com/redhat-developer/vscode-java/blob/master/CONTRIBUTING.md#b-1-setting-up-the-jdt-language-server-in-vs-code" target="_blank" title="https://github.com/redhat-developer/vscode-java/blob/master/CONTRIBUTING.md#b-1-setting-up-the-jdt-language-server-in-vs-code">https://github.com/redhat-developer/vscode-java/blob/master/CONTRIBUTING.md#b-1-setting-up-the-jdt-language-server-in-vs-code</a><br>
and they call it by simply hardcoding the logic and assuming that any test is a JUnit test: <a href="https://github.com/testforstephen/vscode-pde/blob/master/src/extension.ts#L76" target="_blank" title="https://github.com/testforstephen/vscode-pde/blob/master/src/extension.ts#L76">https://github.com/testforstephen/vscode-pde/blob/master/src/extension.ts#L76</a> (also see the <code>isTestMethodNode</code> method).</p>
<p>So, it's more or less as simple as my heuristics now :)</p>



<a name="186377701"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186377701" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> std::Veetaha <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186377701">(Jan 23 2020 at 11:01)</a>:</h4>
<p>Hmm, so does it mean that they support only JUnit test framework?</p>



<a name="186378864"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Runners%20for%20custom%20test%20annotations/near/186378864" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Kirill Bulatov <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Runners.20for.20custom.20test.20annotations.html#186378864">(Jan 23 2020 at 11:20)</a>:</h4>
<p>That is how I've understood it from a quick look at the sources.<br>
But don't believe me, check it yourself, I've posted the related links above.</p>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>